Probabilistic Semi-Supervised Clustering with Constraints

Unsupervised clustering can be significantly improved using supervision in the form of pairwise constraints, i.e., pairs of instances labeled as belonging to same or different clusters. In recent years, a number of algorithms have been proposed for enhancing clustering quality by employing such supervision. Such methods use the constraints to either modify the objective function, or to learn the distance measure. We propose a probabilistic generative model for semi-supervised clustering based on Hidden Markov Random Fields (HMRFs) that provides a principled framework for incorporating supervision into prototype-based clustering. The model allows the use of a broad range of clustering distortion measures, including certain Bregman divergences (e.g., squared Euclidean distance and KL divergence) and directional distances (e.g., cosine distance). We present an algorithm that performs semi-supervised clustering of data by minimizing an objective function derived from the joint probability defined over the HMRF model. Experimental results on several datasets demonstrate the advantages of the proposed framework.